Fix idle range completion scan#832
Conversation
lejeunerenard
left a comment
There was a problem hiding this comment.
This approach is more thorough when completing range requests, but it blocks seek requests when processing large amounts of ranges. Accounting for this seems like it would add more state at the replicator level for tracking where the cursor left off etc.
If we keep _updateNonPrimary's workload small but:
- make it pick random subset of ranges to avoid stacking unfulfillable ranges at the front.
- trigger non-primary updates more often, maybe when inflight goes idle
We can allow it to yield to seeks while being a minor change. I have the following competing PR with the above, which still doesn't pass atm. #841
|
One note, processing can get slow if there are many seeks at once or a very large backlog of ranges. Potentially optimization/perf improrvement can be made here but left it out to make the PR easier |
lejeunerenard
left a comment
There was a problem hiding this comment.
Did another pass. Still need to cover some of the tests but found a potential way that ranges now can be blocked instead of seeks.
lejeunerenard
left a comment
There was a problem hiding this comment.
Only one test i'm not sure what it's purpose is, but otherwise only notes about seeks vs ranges behavior for the PR in the future.
| if (++updates === 1) { | ||
| setImmediate(() => { | ||
| this._inflight._active++ | ||
| this._updateNonPrimary(false).catch(noop) |
There was a problem hiding this comment.
I'm generally not sure about what this test is asserting. But this _updateNonPrimary(false) will always hit the short circuit return at the beginning of the function because _updatesRunning will always be true. It wont queue the updateAll either since the inflight is manually set to be active & the updateAll arg is false.
There was a problem hiding this comment.
I get the test assertion now, but the _updateNonPrimary(false) doesn't exercise any code paths still. I assume this is to emulate how either a data message or an upgrade might have retriggered the _updateNonPrimary() call.
Probably can be removed given the inflight._active++ is the main thing being tested and shows that something is inflight which normally would interrupt the drain without the drain variable storing the original intent to drain.
There was a problem hiding this comment.
_updateNonPrimary(false) does return early, but first sets _updatesQueued = true. That is separate from _inflight._active++: the test covers both new inflight work and another update being queued during the yield. Removing the call would lose coverage of the queued-update race. I clarified this in an inline comment 2407590
There was a problem hiding this comment.
Didn't think about _updatesQueued being set, but it doesn't matter in this case because the drain keeps the inner loop going. The _updatesQueued makes the outer loop go another time, but it does no work because it isnt set to drain by then.
If we want to cover this additional loop in the test case, then we should assert the value of updates at the end. Naively I thought this would be the number of batches from one pass (Math.ceil(totalLength / 64)) plus the additional pass, but I didn't think about how the checkedSinceResolve is updated once a range is resolved. So the actual assertion looks like the following:
const singlePass = Math.ceil(totalLength / 64)
const repassBecauseOfResolving = Math.ceil(availableStart / 64)
const firstPassBatches = singlePass + repassBecauseOfResolving
t.is(updates, firstPassBatches + 1, 'does a second pass because _updateNonPrimary called mid update')Does point out that the work done during a drain is not optimal but since this covers the case where ranges are added, I think it's fine as is. A more efficient approach can be part of refactoring seeks as well.
Also as an aside, the .catch(noop) isn't necessary right? _updateNonPrimary isn't allowed to throw.
There was a problem hiding this comment.
My gut feeling says no about adding extra assertion here for now, since what we are testing that the queued update must not interrupt the current drain, and the extra assertion propose would make it overly strict towards future changes. I did the change fixing the noop catch though.
There was a problem hiding this comment.
Ok then there is no purpose to the _updateNonPrimary call correct? It causes the outer loop to trigger again (by setting _updatesQueued) but the assertions don't cover that without the above addition. Without an assertion, it should be removed as well as the eventFlush() (and inflight._active = 0 could be too).
I could see either way as fine, as they essentially one of two scenarios:
- Testing that
drainfully drains even when inflight blocks are sent when yielding the loop. - The same as above, with the additional coverage of another update queued.
co-authored with AI